The advent of digital multimedia such as digital images, speech/audio, graphics, and video have significantly improved various applications as well as opened up brand new applications due to relative ease by which it has enabled reliable storage, communication, transmission, and, search and access of content. Overall, the applications of digital multimedia have been many, encompassing a wide spectrum including entertainment, information, medicine, and security, and have benefited the society in numerous ways. Multimedia as captured by sensors such as cameras and microphones is often analog, and the process of digitization in the form of Pulse Coded Modulation (PCM) renders it digital. However, just after digitization, the amount of resulting data can be quite significant as is necessary to re-create the analog representation needed by speakers and/or TV display. Thus, efficient communication, storage or transmission of the large volume of digital multimedia content requires its compression from raw PCM form to a compressed representation. Thus, many techniques for compression of multimedia have been invented. Over the years, video compression techniques have grown very sophisticated to the point that they can often achieve high compression factors between 10 and 100 while retaining high psycho-visual quality, often similar to uncompressed digital video.
While tremendous progress has been made to date in the art and science of video compression (as exhibited by the plethora of standards bodies driven video coding standards such as MPEG-1, MPEG-2, H.263, MPEG-4 part2, MPEG-4 AVC/H.264, MPEG-4 SVC and MVC, as well as industry driven proprietary standards such as Windows Media Video, RealVideo, On2 VP, and the like), the ever increasing appetite of consumers for even higher quality, higher definition, and now 3D (stereo) video, available for access whenever, wherever, has necessitated delivery via various means such as DVD/BD, over the air broadcast, cable/satellite, wired and mobile networks, to a range of client devices such as PCs/laptops, TVs, set top boxes, gaming consoles, portable media players/devices, and smartphones, fueling the desire for even higher levels of video compression. In the standards-body-driven standards, this is evidenced by the recently started effort by ISO MPEG in High Efficiency Video coding which is expected to combine new technology contributions and technology from a number of years of exploratory work on H.265 video compression by ITU-T standards committee.
All aforementioned standards employ a general interframe predictive coding framework that involves reducing temporal redundancy by compensating for motion between frames of video by first dividing a frame into blocks and assigning motion vector/s to each block of a frame to be coded, with respect to a past decoded frame; these motion vectors are then transmitted to the decoder and used to generate a motion compensated prediction frame that is differenced with a past decoded frame and coded block by block, often by transform coding. It has been long recognized that at high compression factors at which today's state of the art codecs operate, motion compensated transform coding process may generate coding artifacts that range from coding noise (e.g. mosquito effects), blockiness, bleeding, to ringing. In fact to mitigate some of these coding artifacts, most state of the art codecs include deblocking filters as well as sometimes ringing, as postprocessing filters. Often deblocking is done in-loop as compared to postprocessing that can be done out of loop. Often such deblocking is able to suppress visual quality artifacts but does not compensate for impairments introduced by the codec, nor is such deblocking able to give much coding quality gain.
Recognizing this, over last few years research effort has been underway to design additional filters that can improve both the visual and coding quality. Further, since coding quality can vary a lot depending on content of video sequences, contrast ratio, noise level, and operating bit-rates, it is desirable that the filters are adaptive rather than fixed to get maximum benefit of such a filter to improve prediction and thus help improve compression efficiency. The mathematical procedure used to compute filter coefficients is based on Wiener Hopf equation and computes filter coefficients (e.g. an array of 5×5 or 7×7 or 9×9 coefficients), which are either directly or differentially sent for each frame of every video sequence.
According to the basic theory of adaptive filtering, cross-correlation and autocorrelation matrices are accumulated, from which the optimal Wiener filter can be computed by solving the Wiener Hopf equation as follows.
Let x(n) be the input signal, y(n) be the output, and h(n) represent filter coefficients.
Filter output:
      y    ⁡          (      n      )        =            ∑              k        =        0                    N        -        1              ⁢                  ⁢                  h        ⁡                  (          k          )                    ⁢              x        ⁡                  (                      n            -            k                    )                    
Error signal:e(n)=d(n)−y(n)
Mean Square Error:J=E[e2(n)]
In vector form:
            x      ⁡              (        n        )              =          [                                                  x              ⁡                              (                n                )                                                                                        x              ⁡                              (                                  n                  -                  1                                )                                                                          ⋮                                                              x              ⁡                              (                                  n                  -                  N                  +                  1                                )                                                        ]        and      h    =          [                                                  h              ⁡                              (                0                )                                                                                        h              ⁡                              (                1                )                                                                          ⋮                                                              h              ⁡                              (                                  N                  -                  1                                )                                                        ]                  y      ⁡              (        n        )              =                            h          T                ⁢                  x          ⁡                      (            n            )                              =                                    x            ⁡                          (              n              )                                T                ⁢        h                                                      E            ⁡                          [                                                e                  2                                ⁡                                  (                  n                  )                                            ]                                =                    ⁢                      E            ⁡                          [                                                (                                                            d                      ⁡                                              (                        n                        )                                                              -                                          y                      ⁡                                              (                        n                        )                                                                              )                                2                            ]                                                                    =                    ⁢                                    E              ⁡                              [                                                      d                    2                                    ⁡                                      (                    n                    )                                                  ]                                      -                          2              ⁢                                                          ⁢                              E                ⁡                                  [                                                            d                      ⁡                                              (                        n                        )                                                              ⁢                                                                  x                        ⁡                                                  (                          n                          )                                                                    T                                                        ]                                            ⁢              h                        +                                          h                T                            ⁢                              E                ⁡                                  [                                                            x                      ⁡                                              (                        n                        )                                                              ⁢                                                                  x                        ⁡                                                  (                          n                          )                                                                    T                                                        ]                                            ⁢              h                                                                    =                    ⁢                                    P              d                        -                          2              ⁢                                                          ⁢                              R                dx                T                            ⁢              h                        +                                          h                T                            ⁢                              R                xx                            ⁢              h                                          
Where, Pd is a scalar.
Crosscorrelation row vector:Rdx=E[d(n)x(n)T]
Autocorrelation matrix:Rxx=E[x(n)x(n)T]
To find minimum, the derivative is taken and set to zero as follows:
            ∂              E        ⁡                  [                                    e              2                        ⁡                          (              n              )                                ]                            ∂              h        ⁡                  (          k          )                      =                              -          2                ⁢                                  ⁢                  R          dx          T                    +              2        ⁢                                  ⁢                  R          xx                ⁢        h              =    0  
Solving for h, the Wiener Hopf equation is as follows:h=Rxx−1Rdx 
The Wiener Hopf equation determines optimum filter coefficients in mean square error, and the resulting filter is called the ‘wiener’ filter.
In the above equation, h is the vector of filter coefficients, Rxx is the autocorrelation matrix (or block data of reference frame) and Rdx is a cross-correlation matrix/row vector (between the source frame and reference frame block data).
While the aforementioned mathematical procedure can calculate filter coefficients, there are significant issues in practical integration of this approach in a video coding system. In context of H.264 and H.265 development, a number of proposals have been made to address Picture Based (ALF); Block Based (ALF); Quad-tree Based (QALF); Spatial prediction of coefficients; Postfilter.
A limitation, however, is that the process is computationally expensive (non-separable filters require more computations than separable filters), iterative typically requiring 3-12 full iterations to compute filter coefficients, and further often requires sending of many coefficients (40-80 coefficients) per frame, which even when differentially coded can cost for instance around 300-600 bits per frame of Common Intermediate Format (“CIF”) resolution video, while offering at times limited gains (e.g, due to rounding of coefficients, standard coefficients being good match to the content etc). Further some methods send a macroblock or quad-tree based block map to indicate blocks where such filter is turned ‘on’ or ‘off’; this process can increase overhead bit cost by another 200-400 bits, such that the total bit cost per frame of 500-1000 bits. This makes such a system expensive in overhead bit cost, inflexible, and excessively complex.
Additional information related to adaptive filtering may be found in the following references, each of which is incorporated fully by reference, for all purposes:                T. Chujoh et al, “Block Based Adaptive Loop Filter,” ITU-Telecommunications Standardization Sector, VCEG, VCEG-AI18, Germany, July 2008.        T. Chujoh et al, “Specification and experimental results of Quad-tree-based Adaptive Loop Filter,” ITU-Telecommunications Standardization Sector, VCEG, VCEG-AK22, Japan, April 2009.        